Endoscope systems are widely diffused. According to the endoscope system, a long insertion portion is inserted into a body cavity, an organ in the body cavity is observed on a monitor screen using a solid-state image pick-up device as image pick-up means, and the organ can be examined or diagnosed. Further, ultrasonic endoscope systems are also widespread. According to the ultrasonic endoscope system, an organ in the body cavity is irradiated with ultrasonic waves, and the situation of the organ is observed on a monitor screen using the reflectance or transmittance of the organ with the ultrasonic waves. Thus, the organ can be examined or diagnosed.
The final diagnoses using the above-mentioned endoscope systems mostly depend on the subjectivities of doctors. It is desired that an endoscopic diagnosis support system directly leading to an objective diagnosis using numeric data be realized.
The endoscopic diagnosis support system supports an objective diagnosis with numeric data as follows. Various amounts of feature are calculated from regions of interest (ROI) of an image of a diagnosis target. The image is categorized by threshold processing the amounts of feature or processing the amounts of feature using a statistical or non-statistical discrimination circuit, thus determining a finding or a lesion. The finding or lesion based on the image is provided to a doctor.
The amounts of feature indicate numeric values reflecting various findings of an endoscopic image and are obtained by an image processing method. When a finding related to a color tone, for example, a case where the surface of mucous membrane is red by rubefaction is converted into the amount of feature, R, G, B data constituting an endoscopic image are used, R/(R+G+B) is obtained every pixel, and the average can be used as the amount of feature (the amount of feature is generally called chromaticity). In the recent endoscopic field, as the amount of feature of a color tone reflecting the bloodstream through a gastric mucosa, hemoglobin index obtained by 32 log2(R/G) is generally used.
In addition, findings related to the surface structure of a mucosa in an endoscopic image, for example, the deflation or meandering of a blood vessel observed in a visible blood vessel image, a variation in size of gastric areas, irregularity of gastric area, and the width of a groove between gastric areas are important elements for diagnoses of various diseases. These elements are processed by an image processing method, thus obtaining numeric values as the amounts of feature. Japanese Patent No. 2918162 discloses a method for calculating the above-mentioned amount of feature.
According to a recent endoscopic image processing method, Gabor features calculated using a well-known Gabor filter are processed by a spatial frequency analysis technique, that is improved for the application to endoscopic images, so that the fineness of the surface structure of a mucosa or the orientation of a pattern on the surface structure thereof is digitalized as the amount of feature.
Various amounts of feature obtained from different findings are combined into a feature vector. More complex and correct diagnosis can be supported using the feature vectors. To increase the precision of the endoscopic diagnosis support system, a method for calculating the amount of feature with high precision to digitize a significant finding based on an endoscopic image is very important.
Japanese Unexamined Patent Application Publication No. 10-14864 discloses an example of the above-mentioned endoscopic diagnosis support system and a method for calculating the amount of feature.
Significant findings for diagnoses using endoscopic observations include the shape and size of a lesion, the color tone of a mucosa, a visible blood vessel image, and the surface structure (pattern image including pits) of a mucosa. The present invention relates to an image processing method for a visible blood vessel image and the surface structure of a mucosa among the above findings.
Diseases, diagnosed using a visible blood vessel image as an important finding, include ulcerative colitis. The visible blood vessel image is important to determine the level of inflammation or the degree of remission.
Due to the appearance of a magnifying endoscope in which high image quality is realized and high resolution of a solid-state image pick-up device (CCD, or the like) is also realized, which has the same diameter and operability as those of a normal endoscope and further has a zooming function, a very minute capillary on the surface of a mucosa and the pit pattern of a stomach or a colon can be clearly observed through a recent endoscope system with the magnifying endoscope.
Living tissue can be observed using an endoscope in a clinical examination at the same level as that of a case where a tissue specimen has conventionally been observed using a stereoscopic microscope. New diagnostics using those minute structure observation views are studied actively and are established in fields related to digestive tracts and bronchial tubes.
The following examples are given:
Diagnosis of an esophageal neoplastic lesion (adenoma, cancer) based on a pattern change in interpapillary capillary loop (IPCL) on an esophageal mucosa disclosed in Reference Document 1 (H. Inoue, MAGNIFICATION ENDOSCOPY IN THE SOPHAGUS AND STOMACH, Digestive Endoscopy, JAPAN ASTROENTEROLOGICAL ENDOSCOPY SCOCIETY, Vol. 13, Supplement, July 2001, pp. S40-S41);
Diagnosis of an infection with Helicobacter pylori based on observations of gastric collecting venules disclosed in Reference Document 2 (K. Yagi, ENDOSCOPIC FEATURES AND MAGNIFIED VIEWS OF THE CORPUS IN THE HELICOBACTER PYLORI-NEGATIVE STOMACH, Digestive Endoscopy, JAPAN GASTROENTEROLOGICAL ENDOSCOPY SOCIETY, Vol. 13, Supplement, July 2001, pp. S34-S35);
Diagnosis of a classification of mucosal atrophy and neoplastic lesion based on observations of gastric microvasculatures disclosed in Reference Document 3 (K. Yao et al., MICROGASTROSCOPIC FINDINGS OF MUCOSAL MICROVASCULAR ARCHITECTURE AS VISUALIZED BY MAGNIFYING ENDOSCOPY, Digestive Endoscopy, JAPAN GASTROENTEROLOGICAL ENDOSCOPY SOCIETY, Vol. 13, Supplement, July 2001, pp. S27-S33);
Diagnosis of a neoplastic lesion of a colon using pit pattern classification disclosed in Reference Document 4 (Shin-ei Kudo, Depression Type Early Colonic Cancer, Nihon Medical Center, 1996, pp. 33-40); and
Diagnosis of a bronchial infection and cancer based on observations of a microvasculature network in bronchi disclosed in Reference Document 5 (Kiyoshi Shibuya et al., Endoscopic Observations of Bronchial Dysplasia using Magnifying Bronchial Video Scope, Vol. 22, No. 8, December 2000, pp. 613-616).
On the other hand, those diagnoses based on endoscopic findings depend on subjective determinations of doctors. Disadvantageously, a variation in experience and knowledge between doctors may result in different diagnoses. It is desired that quantitative and objective diagnosis support information be provided using image processing.
More specifically, the pattern of a blood vessel or pits is extracted from an image, the form, size, uniformity, or regularity of the pattern is digitized (called the amount of feature) by various image processing techniques, and the amount of feature is processed by a discrimination circuit such as a linear discriminant function or a neural network. Thus, objective diagnosis support can be realized.
Japanese Patent No. 2918162, to the same assignee, discloses an image processing and analysis method using a binarization process.
However, the application of the image processing method with structure extraction by binarization to an endoscopic image has the following disadvantages.
Endoscopic images are obtained with various observation distances and angles. Further, in many cases, an observation subject has a curved form. Therefore, an endoscopic image has a large variation in brightness. In observing a pit pattern of a colon, generally, dye or a coloring agent, typified by indigo carmine or crystal violet, is sprayed in order to make pits clear. The density of dye may be varied or the dye may be sprayed unevenly (dye may be remained in interstitial portions other than pits).
Threshold processing is generally executed in the binarization process. For the above reason, however, it is difficult to use a fixed threshold value. Even when a threshold value is changed every image (it is studied in the character recognition field), the extraction result is changed every local area of the image because a variation in brightness or the spray condition of dye. Disadvantageously, extraction leakage may occur.
The present invention is made in consideration of the above disadvantages. It is an object of the present invention to provide an image processing method whereby a structural component such as a blood vessel image or a pit pattern serving as an extraction target can be favorably extracted in an (endoscopic) image.
Another object of the present invention is to provide an image processing method whereby a structural component such as a blood vessel image or a pit pattern serving as an extraction target can be favorably extracted in an (endoscopic) image irrespective of image shooting conditions.
Further, still another object of the present invention is to provide an image processing method for calculating the amount of feature to realize high-precision diagnosis support information.